Electronic devices having moisture-insensitive optical touch sensors

ABSTRACT

An electronic device may have a touch sensitive display that is insensitive to the presence of moisture. An array of pixels in the display may be used to display images. A display cover layer may overlap the array of pixels. A light source may illuminate an external object such as a finger of a user when the object contacts a surface of the display cover layer. This creates scattered light that may be detected by an array of light sensors. The light source may supply light to an edge of the display cover layer at an angle that ensures total internal reflection within the display cover layer is sustained across the display cover layer even when the display cover layer is immersed in water or otherwise exposed to moisture.

FIELD

This relates generally to electronic devices, and, more particularly, toelectronic devices with touch sensors.

BACKGROUND

Electronic devices such as tablet computers, cellular telephones, andother equipment are sometimes provided with touch sensors. For example,displays in electronic devices are often provided with capacitive touchsensors to receive touch input. It can be challenging to operate suchsensors in the presence of moisture.

SUMMARY

An electronic device may have a touch sensitive display that isinsensitive to the presence of moisture. The display may have atwo-dimensional optical touch sensor such as a direct illuminationoptical touch sensor or a total internal reflection touch sensor. Theoptical touch sensor may be used to gather touch input while theelectronic device is immersed in water or otherwise exposed to moisture.

An array of pixels in the display may be used to display images. Adisplay cover layer may overlap the array of pixels. A light source mayilluminate an external object such as a finger of a user when the objectcontacts a surface of the display cover layer. This creates scatteredlight that may be detected by an array of light sensors. The lightsource may supply light to an edge of the display cover layer at anangle that ensures total internal reflection is sustained within thedisplay cover layer when the display cover layer is immersed in water orotherwise exposed to moisture.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an illustrative electronic device inaccordance with an embodiment.

FIG. 2 is a perspective view of an illustrative electronic device inaccordance with an embodiment.

FIG. 3 is a cross-sectional side view of an illustrative electronicdevice in accordance with an embodiment.

FIG. 4 is a top view of an illustrative array of pixels for anelectronic device in accordance with an embodiment.

FIGS. 5 and 6 are cross-sectional side views of illustrative pixelarrays for electronic devices in accordance with embodiments.

FIG. 7 is a cross-sectional side view of an illustrative optical touchsensor arrangement in accordance with an embodiment.

FIG. 8 is a cross-sectional side view of an illustrative optical touchsensor arrangement based on total internal reflection in accordance withan embodiment.

FIGS. 9, 10, and 11 are cross-sectional side views of illustrativedisplay and sensor arrangements with different numbers of pixel layersin accordance with embodiments.

FIG. 12 is a cross-sectional side view of an illustrative pixel withthin-film circuit structures in accordance with an embodiment.

FIG. 13 is a cross-sectional side view of illustrative pixels formedfrom crystalline semiconductor dies in accordance with an embodiment.

FIG. 14 is a cross-sectional side view of an illustrative light sourceconfigured to emit light into a display cover layer through anindex-matching structure in accordance with an embodiment.

FIGS. 15, 16, 17, 18, and 19 are cross-sectional side views ofillustrative apertures that may be placed over light detectors inaccordance with embodiments.

FIGS. 20 and 21 are timing diagrams showing how light-emitting pixelsand light-sensing pixels may operate in an electronic device inaccordance with embodiments.

FIG. 22 is a cross-sectional side view of an illustrative pixel array ofthe type that may include light sensing pixels in accordance with anembodiment.

FIG. 23 is a cross-sectional side view of an illustrative display withan optical touch sensor formed from components located above and/orbelow an array of image pixels in accordance with an embodiment.

FIG. 24 is a cross-sectional side view of an illustrative display withbacklight unit pixels that may be used in forming optical touch sensorstructures in accordance with an embodiment.

DETAILED DESCRIPTION

A schematic diagram of an illustrative electronic device that mayinclude an optical touch sensor is shown in FIG. 1. Electronic device 10of FIG. 1 may be a computing device such as a laptop computer, acomputer monitor containing an embedded computer, a tablet computer, acellular telephone, a media player, or other handheld or portableelectronic device, a smaller device such as a wristwatch or other deviceworn on a user's wrist, a pendant device, a headphone or earpiecedevice, a head-mounted device such as eyeglasses, goggles, or otherequipment worn on a user's head, or other wearable or miniature device,a television, a computer display that does not contain an embeddedcomputer, a gaming device, a navigation device, an embedded system suchas a system in which electronic equipment with a display is mounted in akiosk or automobile, equipment that implements the functionality of twoor more of these devices, or other electronic equipment. Illustrativeconfigurations in which device 10 is a portable device such as awristwatch, cellular telephone, or tablet computer and, moreparticularly, a portable device that is water resistant or waterproofmay sometimes be described herein as an example.

As shown in FIG. 1, electronic device 10 may have control circuitry 16.Control circuitry 16 may include storage and processing circuitry forsupporting the operation of device 10. The storage and processingcircuitry may include storage such as hard disk drive storage,nonvolatile memory (e.g., flash memory or otherelectrically-programmable-read-only memory configured to form a solidstate drive), volatile memory (e.g., static or dynamicrandom-access-memory), etc. Processing circuitry in control circuitry 16may be used to control the operation of device 10. The processingcircuitry may be based on one or more microprocessors, microcontrollers,digital signal processors, baseband processors, power management units,audio chips, application specific integrated circuits, etc. Controlcircuitry 16 may include communications circuitry for supporting wiredand/or wireless communications between device 10 and external equipment.For example, control circuitry 16 may include wireless communicationscircuitry such as cellular telephone communications circuitry andwireless local area network communications circuitry.

Input-output circuitry in device 10 such as input-output devices 12 maybe used to allow data to be supplied to device 10 and to allow data tobe provided from device 10 to external devices. Input-output devices 12may include buttons, joysticks, scrolling wheels, touch pads, key pads,keyboards, microphones, speakers, tone generators, haptic outputdevices, cameras, light-emitting diodes and other status indicators,data ports, etc. A user can control the operation of device 10 bysupplying commands through input-output devices 12 and may receivestatus information and other output from device 10 using the outputresources of input-output devices 12.

Input-output devices 12 may include one or more displays such as display14. Display 14 may be an organic light-emitting diode display, a displayformed from an array of crystalline semiconductor light-emitting diodedies, a liquid crystal display, or other display. Display 14 may be atouch screen display that includes an optical touch sensor for gatheringtouch input from a user. The optical touch sensor may be configured tooperate even when device 10 is immersed in water or otherwise exposed tomoisture. If desired, the optical touch sensor may also be configured tooperate when a user is wearing gloves, which might be difficult orimpossible with some capacitive touch sensors. Moreover, because theoptical touch sensor operates optically, the touch sensor is notimpacted by grounding effects that might impact the operation ofcapacitive touch sensors.

As shown in FIG. 1, input-output devices 12 may include sensors 18.Sensors 18 may include touch sensors. Touch sensors may be provided fordisplay 14 and/or other portions of device 10 and may be formed from anarray of capacitive touch sensor electrodes, acoustic touch sensorstructures, resistive touch components, force-based touch sensorstructures, light-based touch sensor structures, or other suitable touchsensor arrangements. Illustrative optical touch sensor arrangements fordevice 10 (e.g., for display 14 of device 10) are sometimes describedherein as an example.

Sensors 18 may include capacitive sensors, light-based proximitysensors, magnetic sensors, accelerometers, force sensors, touch sensors,temperature sensors, pressure sensors, inertial measurement units,accelerometers, gyroscopes, compasses, microphones, radio-frequencysensors, three-dimensional image sensors (e.g., structured light sensorswith light emitters such as infrared light emitters configured to emitstructured light and corresponding infrared image sensors,three-dimensional sensors based on pairs of two-dimensional imagesensors, etc.), cameras (e.g., visible light cameras and/or infraredlight cameras), light-based position sensors (e.g., lidar sensors),monochrome and/or color ambient light sensors, and other sensors.Sensors 18 such as ambient light sensors, image sensors, opticalproximity sensors, lidar sensors, optical touch sensors, and othersensors that use light and/or components that emit light such as statusindicator lights and other light-emitting components may sometimes bereferred to as optical components.

A perspective view of an illustrative electronic device of the type thatmay include an optical touch sensor is shown in FIG. 2. In the exampleof FIG. 2, device 10 includes a display such as display 14 mounted inhousing 22. Display 14 may be a liquid crystal display, a light-emittingdiode display such as an organic light-emitting diode display or adisplay formed from crystalline semiconductor light-emitting diode dies,or other suitable display. Display 14 may have an array of image pixelsextending across some or all of front face F of device 10 and/or otherexternal device surfaces. The array of image pixels may be rectangularor may have other suitable shapes. Display 14 may be protected using adisplay cover layer (e.g., a transparent front housing layer) such as alayer of transparent glass, clear plastic, sapphire, or other clearlayer. The display cover layer may overlap the array of image pixels.

Housing 22, which may sometimes be referred to as an enclosure or case,may be formed of plastic, glass, ceramics, fiber composites, metal(e.g., stainless steel, aluminum, etc.), other suitable materials, or acombination of any two or more of these materials. As shown in thecross-sectional side view of device 10 of FIG. 3, housing 22 and display14 may separate an interior region of device 10 such as interior region30 from an exterior region surrounding device 10 such as exterior region32. Housing 22 may be formed using a unibody configuration in which someor all of housing 22 is machined or molded as a single structure or maybe formed using multiple structures (e.g., an internal frame structure,one or more structures that form exterior housing surfaces, etc.). Ifdesired, a strap may be coupled to a main portion of housing 22 (e.g.,in configurations in which device 10 is a wristwatch or head-mounteddevice). Internal electrical components 36 (e.g., integrated circuits,discrete components, etc.) for forming control circuitry 16 andinput-output devices 12 may be mounted in interior 30 of housing 22(e.g., on one or more substrates such as printed circuit 38). In someconfigurations, components 36 may be attached to display 14 (e.g.,circuitry may be mounted to the surface of display 14). To obtain touchinput from a user's fingers or other external object (see, e.g., userfinger 34), display 14 may include a touch sensor such as an opticaltouch sensor (e.g., a two-dimensional optical touch sensor that gathersinformation on the XY location of a user's finger or other externalobject when that object touches the surface of display 14).

Display 14 may include a display panel such as display panel 14P thatcontains pixels P covered by display cover layer 14CG. The pixels ofdisplay 14 may cover all of the front face of device 10 or display 14may have pixel-free areas (e.g., notches, rectangular islands, inactiveborder regions, or other regions) that do not contain any pixels.Pixel-free areas may be used to accommodate an opening for a speaker andwindows for optical components such as image sensors, an ambient lightsensor, an optical proximity sensor, a three-dimensional image sensorsuch as a structured light three-dimensional image sensor, a cameraflash, an illuminator for an infrared image sensor, an illuminator for athree-dimensional sensor such as a structured light sensor, atime-of-flight sensor, a lidar sensor, etc.

FIG. 4 is a top view of an array of illustrative pixels P in displaypanel (display) 14P. As shown in FIG. 4, pixels P may include imagepixels such as pixel P-1 that are used in presenting images for a userof device 10. Image pixels in display 14 may, for example, include arectangular array of red, green, and glue light-emitting diodes orbacklight red, green, and blue liquid crystal display pixels forpresenting color images to a user.

Pixels P may also contain optical touch sensor pixels such as pixel P-2.Optical touch sensor pixels may include pixels that serve as lightdetectors and/or light emitters. Emitted light that reflects from auser's finger on the surface of display 14 may be detected using thelight detectors, thereby determining the location of the user's finger.If desired, diodes or other components may be used to form pixels thatcan be operated both as image pixels and as touch sensor pixels. Whenused as touch sensor pixels, image pixels can be configured to emitoptical touch sensor illumination and/or to detect optical touch sensorlight. For example, a display emitter can be used to produce image lightfor a display while also being used to produce optical touch sensorillumination, and/or while also being used to serve as a photodetectorfor an optical touch sensor.

Image pixels such as pixels P-1 and/or optical touch sensor pixels P-2may have any suitable pitch. For example, image pixels may have adensity that is sufficient to display high-quality images for a user(e.g., 200-300 pixels per inch or more, as an example), whereas opticaltouch sensor pixels may, if desired, have a lower density (e.g., lessthan 200 pixel per inch, less than 50 pixels per inch, less than 20pixels per inch, etc.).

Image pixels emit visible light for viewing by a user. For example, in acolor display, image pixels may emit light of different colors of imagelight such as red, green, and blue light, thereby allowing display 14 topresent color images. Optical touch sensor pixels may emit and/or detectvisible light and/or infrared light (and/or, if desired, ultravioletlight).

In some configurations, optical touch sensor light for illuminating auser's fingers passes directly through the thickness of display coverlayer 14CG from its interior surface to its exterior surface. Opticaltouch sensors in which light that illuminates the user's fingers passesoutwardly from light sources such as light-emitting pixels in displaypanel 14P directly through the thickness of display cover layer 14CGbefore being backscattered in the reverse (inward) direction to thelight detectors of the optical touch sensors may sometimes be referredto herein as direct illumination optical touch sensors.

In other configurations, light for an optical touch sensor may beprovided using edge-coupled light-emitting diodes or other light sourcesthat emit light into the edge surface of display cover layer 14P that isthen guided within layer 14CG in accordance with the principal of totalinternal reflection. For example, a light-emitting diode may emit lightinto the righthand edge of display cover layer 14CG that is guided fromthe righthand edge of display cover layer 14CG to the opposing lefthandedge of display cover layer 14CG within the light guide formed bydisplay cover layer 14CG. In this way, light may be guided laterallyacross layer 14CG in the absence of contact from a user's finger. When auser's finger touches the surface of layer 14CG, total internalreflection can be locally defeated. This local frustration of totalinternal reflection scatters light inwardly toward the light detectorsof the optical touch sensor. Optical touch sensors that are based onlocally defeating total internal reflection may sometimes be referred toherein as total internal reflection optical touch sensors. If desired,objects other than the fingers of users (e.g., a computer stylus, aglove, and/or other external objects with appropriate opticalproperties) may also locally defeat total internal reflection, therebyallowing the optical touch sensors to function over a wide range ofoperating environments.

Pixels P that emit light and pixels P that detect light in display panel14P may be formed using shared structures and/or structures that areseparate from each other. These structures may be located in the sameplane (e.g., as part of a single layer of pixels on a single substrate)and/or may include components located in multiple planes (e.g., inarrangements in which some components are formed in a given layer andother components are formed in one or more additional layers aboveand/or below the given layer).

Consider, as an example, an optical touch sensor that contains an arrayof photodetectors formed from reverse-biased diodes. These diodes may bededicated photodetectors or may be light-emitting didoes that serve aslight detectors when reverse biased and that serve as light sources whenforward biased. Light sources in the optical touch sensor may includevisible light sources (e.g., visible light sources dedicated to use inthe optical touch sensor or visible light sources that also serve asimage pixels) and/or may include infrared light sources. Light-emittingpixels for the optical touch sensor may be formed from light-emittingdiodes (e.g., dedicated light-emitting diodes or diodes that serve aslight-emitting diodes when forward biased and that serve asphotodetectors when reversed biased). Light-emitting pixels may also beformed from pixels P that are backlit with light from a backlight unitto form backlit pixels (e.g., backlit liquid crystal display pixels). Ingeneral, any type of photodetector signal processing circuitry may beused to detect when a photodetector has received light. For example,photodetectors may be configured to operate in a photoresistor mode inwhich the photodetectors change resistance upon exposure to light andcorresponding photodetector signal processing circuitry may be used tomeasure the changes in photodetector resistance. As another example, thephotodetectors may be configured to operate in a photovoltaic mode inwhich a voltage is produced when light is sensed and correspondingphotodetector signal processing circuitry may be used to detect thevoltage signals that are output from the photodetectors. Semiconductorphotodetectors may be implemented using phototransistors or photodiodes.Other types of photosensitive components may be used, if desired.

FIG. 5 is a cross-sectional side view of an illustrative display havingan array of pixels P that are not backlit. Pixels P of FIG. 5 mayinclude light-emitting diodes (e.g., organic light-emitting diodes suchas thin-film organic light-emitting diodes and/or light-emitting diodesformed from crystalline semiconductor light-emitting diode dies). Duringoperation, image pixels formed from the light-emitting diodes maypresent an image on display 14 that is visible to a user such as viewer40 who is viewing display 14 in direction 42.

FIG. 6 is a cross-sectional side view of an illustrative display havingan array of pixels P that are backlit using backlight unit 44. Backlightunit 44 may include one or more strips of light-emitting diodes thatemit light into a backlight unit light guide layer (e.g., a clearoptical film with light-scattering structures). As the emitted lightpropagates through the light guide layer, the scattered light serve asbacklight illumination for pixels P (e.g., liquid crystal displaypixels). In another illustrative configuration, backlight unit 44 is adirect lit backlight unit that contains an array of backlightlight-emitting diodes that provide backlight (e.g., an array-typebacklight unit that supports local dimming functionality).

FIG. 7 is a cross-sectional side view of an illustrative display with adirect illumination optical touch sensor. As shown in FIG. 7, visibleand/or infrared light sources associated with display panel 14P may emitillumination 46 that travels directly through display cover layer 14CGfrom its inner surface to its outer surface, thereby illuminating anexternal object contacting the surface of display 14 such as finger 34.This creates localized backscattered light 48 that propagates in theinward (−Z) direction and that is detected by photodetectors associatedwith display panel 14P that are directly below finger 34. In this way,the optical touch sensor can determine the lateral position (XYlocation) of finger 34.

FIG. 8 is a cross-sectional side view of an illustrative display with atotal internal reflection optical touch sensor. As shown in FIG. 8,display 14 may include display cover layer 14CG and display panel 14P.Image pixels in panel 14P may display images that are viewable by aviewer through display cover layer 14CG. The outermost surface ofdisplay panel 14P may be separated from the opposing innermost surfaceof display cover layer 14CG by layer 50. Layer 50 may be formed fromair, liquid, polymer (e.g., polymer adhesive such as optically clearadhesive, pressure sensitive adhesive, other polymer materials, etc.),glass, other materials, and/or combinations of these materials. Light 46maybe coupled into layer 14CG through the sidewalls of layer 14CG (e.g.,at the righthand edge surface at the peripheral of display cover layer14CG in the example of FIG. 8).

Any suitable optical coupling structures may be used to direct light 46into display cover layer 14CG. In the example of FIG. 8, light 46 isemitted by a light source such as light source 52. Light source 52 maybe a light-emitting diode such as a visible or infrared light-emittingdiode or a visible or infrared laser diode. Collimator 54 may be used tocollimate the emitted light from light source 52 (e.g., to form a beamof light with parallel light rays). A prism such as prism 56 or otheroptical coupler may be coupled between collimator 54 and display coverlayer 14CG. Prism 56 may, for example, be mounted to the edge of displaycover layer 14CG to help direct light into the edge of display coverlayer 14CG. During operation, optical coupling structures such ascollimator 54 and a prism or other optical coupler may be used to couplelight 46 that is emitted from light source 52 into the interior ofdisplay cover layer 14CG in a beam that is oriented at a desired anglerelative to the surfaces of layer 14CG (e.g., at an angle A with respectto surface normal n of display cover layer 14CG). At this angle A, light46 will propagate within layer 14CG in accordance with the principal oftotal internal reflection unless total internal reflection is locallydefeated by the presence of finger 34 on the outer surface of layer14CG.

Angle A is selected (and the materials used for layer 14CG and layer 50are selected) so that light 46 will reflect from the innermost surfaceof layer 14CG in accordance with the principal of total internalreflection. Layer 14CG may, as an example, have a refractive index n1(e.g., 1.5 for glass or 1.76 for sapphire as examples), whereas layer 50may have a refractive index n2 that is less than n1 (e.g., less than 1.5when layer 14CG is glass or less than 1.76 when layer 14CG is sapphire).The refractive index difference between n1 and n2 may be at least 0.05,at least 0.1, at least 0.2, or other suitable value).

Angle A is also selected so that light 46 will reflect from theuppermost surface of layer 14CG in accordance with the principal oftotal internal reflection (in the absence of finger 34). In someenvironments, device 10 will be immersed in water 60 or otherwiseexposed to moisture (rain droplets, perspiration, fresh or salt watersurrounding device 10 when a user is swimming, etc.). Angle A ispreferably selected to ensure that the presence of water 60 will notdefeat total internal reflection while ensuring that the presence offinger 34 will locally defeat total internal reflection and therebyproduce localized scattered light 48 for detection by the nearbyphotodetectors of the optical touch sensor. This allows the totalinternal reflection optical touch sensor to operate whether or not thesome or all of the surface of display 14 is immersed in water orotherwise exposed to moisture.

Consider, as an example, a first illustrative scenario in which layer14CG is formed from a material with a refractive index of 1.5 (e.g.,glass). Finger 34 may be characterized by a refractive index of 1.55.Water 60 may be characterized by a refractive index of 1.33. Layer 50may have a refractive index of less than 1.5. In this first scenario,total internal reflection at the upper surface of layer 14CG when water60 is present is ensured by the selection of a material for layer 14CGwith a refractive index greater than water and by selecting angle A tobe greater than the critical angle at the upper surface of layer 14CG(in this example, greater than 62.46°, which is the critical angleassociated with total internal reflection at the glass/water interface).To ensure total internal reflection is sustained at the lower surface oflayer 14CG, the selected value of A should be greater than the criticalangle associated with the lower interface. If, as an example, layer 50is formed from a material with a refractive index of 1.33 (the same aswater) or less, the critical angle associated with the lower interfacewill be at least 62.46°, so A should be greater than 62.46°. If, on theother hand, layer 50 is formed from a material with a refractive indexbetween 1.33 and 1.5, the critical angle at the lower interface will beincreased accordingly and the angle A should be increased to besufficient to ensure total internal reflection at the lower interface.Regardless of which value is selected for angle A, total internalreflection will be supported at both the lower and upper surfaces oflayer 14CG (whether layer 14CG is in air or immersed in water), so longas finger 34 is not present. Because finger 34 has a refractive index(1.55) that is greater than that of layer 14CG (which is 1.5 in thisfirst scenario), whenever finger 34 is present on the upper surface oflayer 14CG, total internal reflection will be defeated at finger 34,resulting in scattered light 48 that can be detected by the lightdetectors of the total internal reflection optical touch sensorassociated with display 14.

The refractive index of layer 14CG need not be less than the refractiveindex of finger 34. Consider, as an example, a second illustrativescenario in which layer 14CG is formed from a crystalline material suchas sapphire with a refractive index of 1.76. In this second scenario,the angle A should be selected to be both: 1) sufficiently high toensure that total internal reflection is sustained at the upper (andlower) surfaces of layer 14CG in the absence of finger 34 (even if water60 is present) and 2) sufficiently low to ensure that total internalreflection at the upper surface will be locally defeated when finger 34is touching the upper surface to provide touch input. Total internalreflection at the upper surface may be ensured by selecting a value of Athat is greater than the critical angle associated with a sapphire/waterinterface (e.g., the value of angle A should be greater thanarcsin(1.33/1.76), which is 49.08°). Total internal reflection at thelower interface is ensured by selecting a material for layer 50 that hasan index of refraction of 1.33 or less (in which case A may still begreater than 49.08°) or by selecting a material for layer 50 that has alarger index (but still less than 1.55) and adjusting the value of Aupwards accordingly. To ensure that total internal reflection at theupper surface can be defeated locally by finger 34, the value of angle Ashould be less than the critical angle associated with a sapphire/fingerinterface (e.g., less than arcsin(1.55/1.76), which is 61.72°). Thus, inscenarios in which the refractive index of layer 14CG is greater thanthe refractive index of finger 34, there will be a range of acceptablevalues for A bounded by a lower limit (e.g., 49.08° in this example) andan upper limit (e.g., 61.72° in this example).

In display 14 (e.g., in display panel 14P), the image pixels that areused in displaying images for a user (e.g., the red, blue, and greenpixels in a color display) and/or the optical touch sensor pixels (e.g.,light emitters and/or detectors for implementing a direct illuminationand/or total internal reflection optical touch sensor) may beimplemented using one or more layers of pixels, as shown in thecross-sectional side view of the illustrative displays of FIGS. 9, 10,and 11. FIG. 9 is an illustrative arrangement for display panel 14P thathas a single layer of pixels P. In FIG. 10, two layers of pixels P areused in display panel 14P. The diagram of FIG. 11 shows how displaypanel 14P may, if desired, have three or more layers of pixels P. Ingeneral, optical touch sensor pixels may be located in the same layer asimage sensor pixels and/or may be located in a layer that is above orbelow the image sensor pixels.

Pixels P of FIGS. 9, 10, and 11 may include image pixels and/or opticaltouch sensor pixels. In some arrangements, pixels P may includebacklight pixels that supply backlight illumination in a local dimmingbacklight unit. The pixels P in different layers may have the same pitchor different pitches. As an example, there may be more image pixels perinch than optical touch sensor pixels. Thin-film structures and/ordiscrete devices may be used in forming pixels P. In some embodiments ofdisplay panel 14P (e.g., displays with a total internal reflectionoptical touch sensor), light sources for the optical touch sensor may beconfigured to provide edge illumination (see, e.g., light source 52 ofFIG. 8) in addition to or instead of using light sources in pixels P.

FIG. 12 is a cross-sectional side view of an illustrative display panelwith thin-film optical structures for forming pixels P. Pixel P ofdisplay panel 14P of FIG. 12 may be a thin-film diode (e.g., an organiclight-emitting diode and/or a thin-film organic photodetector or otherthin-film photodetector formed from a reverse biased thin-film diode).As shown in FIG. 12, panel 14P may have a substrate 62. Substrate 62 maybe formed from glass, polymer, and/or other materials. One more layersof material such as thin-film layers 64 may be formed on substrate 62.Layers 64 may include buffer layers, dielectric layers and layers ofmetal traces for forming an interconnect stack, thin-film semiconductorlayers for diodes, thin-film transistors, capacitors, and otherthin-film circuitry, organic layers (e.g., organic emissive layers),encapsulation layers (e.g., encapsulation layers formed from siliconoxide, silicon nitride, other inorganic dielectric materials, and/ororganic dielectric encapsulation materials), and/or other layers. In theexample of FIG. 12, layers 64 include a patterned layer (e.g., apatterned metal layer) forming anode 66, organic layer(s) 68 such asemissive layers for a light-emitting diode, patterned pixel definitionlayer 70 (e.g., a dark polymer layer that has openings for respectiveanodes 66 for diodes, which each also have an opposing overlappingtransparent cathode such as a global cathode that overlaps layer 70 andlayers 68), and encapsulation layer(s) 72. Other thin-film circuitry maybe formed on substrates such as substrate 62 to form display panelstructures (e.g., one or more layers of pixels P for panel 14P), ifdesired.

FIG. 13 is a cross-sectional side view of an illustrative display panelwith crystalline semiconductor dies 74 on substrate 62. Dies 74 mayinclude visible and/or infrared light-emitting diodes for pixels Pand/or photodetectors (e.g., diodes that may be reverse biased).Substrate 62 of FIG. 13 may be a flexible or rigid layer of polymerforming a flexible or rigid printed circuit or may be formed from othersubstrate materials.

Display panels 14P of FIGS. 12 and 13 are illustrative. In general, oneor more layers of pixels P may be formed using one or more display panelstructures (e.g., stacked panels) of the types shown in FIG. 12 (e.g.,one or more thin-film panels and/or one or more panels of dies mountedon printed circuits) and/or FIG. 13. To help ensure sufficienttransparency when layers of pixels overlap each other, the upperlayer(s) of pixels may have transparent areas. For example, in ascenario in which pixels P of FIG. 12 overlaps infrared light sourcesfor a touch sensor, anodes 66 may be configured to be sufficientlytransparent to infrared wavelengths to allow infrared light from theinfrared light sources to pass through anodes 66. In this type ofarrangement, the pixels P in FIG. 12 may be image pixels that emitvisible light. Infrared light sources and detectors (e.g., sources anddetectors formed using diodes 74 of FIG. 13) may be located below theimage pixels (as an example). Transparency in the upper layer(s) of adisplay with stacked layers of pixels P may be also be provided byforming layers such as layer 70, other layers 64 of FIG. 12, and/orlayers such as substrate 62 of FIG. 13 from transparent material, byforming holes in substrate layers, pixel definition layers, and/or otherdisplay layers, by selectively omitting some or all of the anodes orother structures in certain pixels P to create transparent windowregions in a layer of pixels, etc.

As described in connection with FIG. 8, one or more light-emittingdiodes or other light sources such as light source 52 may be used (withan optical coupler) to emit a beam of light 46 into display cover layer14CG at a desired angle A in a total internal reflection optical touchsensor. If desired, light 46 may be coupled into layer 14CG for totalinternal reflection using one or more overlapped light sources 52 (e.g.,an array of infrared and/or visible light sources such as light-emittingdiodes and/or laser diodes that lie below an array of image pixels inpanel 14P). As shown in FIG. 14, for example, display panel 14P may havean array of light sources 52 each of which emits light 46′ in avertically oriented cone. Index-matching structures such as layer 78 maybe provided with a refractive index value equal to or close to that oflayer 14CG to help couple emitted light from each source 52 into layer14CG and/or may include gratings or other optical coupling structures.The lowermost surface of layer 78 may, if desired, be angled withrespect to surface normal n of layer 14CG (e.g., for form a prism)and/or may contact source 52 to help receive light 46′ from source 52without undesired reflections. The configuration of FIG. 14 isillustrative. Mask 76 may be formed on layer 78. Mask 76 may have aring-shaped opening 80 or other opening that restricts the angularorientation of light 46′ as light 46′ passes through mask 76 and layer78 into layer 14CG. In this way, the light from source 52 ischaracterized by rays of light 46 in layer 14CG that are oriented at adesired angle A with respect to surface normal n to support totalinternal reflection in layer 14CG in the absence of finger 34. Lightsources such as light source 52 of FIG. 14 may be pixels P that arelocated in, above, and/or below image pixels in panel 14P. If desired,light sources such as light source 52 of FIG. 14 may be formed frommultiple light sources (e.g., light source stacked on top of each otheror mounted side-by-side on a shared substrate). In this type ofarrangement, each of the multiple light sources may be optimized for aparticular function. for example, one light source may be configured toproduce display illumination and another may be configured to producecollimated total internal reflection illumination for the optical touchsensor.

It may be desirable to restrict the acceptance angles associated with agiven light-detecting pixel. For example, it may be desirable to providephotodetector pixels in an optical touch sensor with angular filtersthat cause the photodetector pixels to be primarily or exclusivelyresponsive to scattered light rays that are perpendicular to the surfacenormal n of layer 14CG (e.g., light rays that are traveling directlyinward from layer 14CG after scattering from a user's finger 34). Inthis way, the impact of noise from stray light may be reduced.

Increased sensitivity to light of a desired angular orientation may beachieved using angle-of-acceptance light filters. Consider, as anexample, the arrangement of FIG. 15. As shown in FIG. 15, angular filter82 may be formed from mask 88 on a transparent layer 84. An optionallens such as lens 86 may overlap and be aligned with opening 90 in mask88. Mask 88 may be formed from black ink, metal, or other opaque maskingmaterials. Opening 90 may be a circular aperture or other gap in theopaque layer of mask 88. Transparent layer 84 may be one of the layersin panel 14P such as an encapsulation layer or other clear dielectriclayer.

In the configuration of FIG. 15, only scattered rays of light 48 thatare propagating perpendicular to layer 14CG (e.g., parallel to surfacenormal n of layer 14CG) will pass through opening 90 after passingthrough lens 86. Off-axis light rays such as off-axis scattered lightray 48′ of FIG. 15 will be blocked by mask 88. A light detecting pixelfor the optical touch sensor may be located under opening 90 inalignment with opening 90, so that on-axis light can be detected.

If desired, filters such as filter 82 may be configured to pass only tooff-axis light of a desired angle (see, e.g., filter 82 of FIG. 16,which may pass only off-axis light rays 48″ to an overlappedlight-detector due to the lateral offset D between the center of lens 86and the center of opening 90). Off-axis filters such as filter 82 ofFIG. 16 may be used in panel 14P, on edge surfaces of layer 14CG, and/orat other locations in device 10 associated with an optical touch sensor(e.g., to help improve detection sensitivity by gathering only rays oflight at particular angles associated with finger-scattered light,guided light, etc.).

Masks such as mask 88 of FIGS. 15 and 16 may be formed on any suitabletransparent layer(s) 84. In the example of FIG. 17, a single mask layeris being used to form mask 88 and that single mask layer is on the topsurface of transparent layer 84. In the example of FIG. 18, mask 88 hasfirst and second mask layers 88′ on opposing upper and lower surfaces oflayer 84 (e.g., an encapsulation layer or other transparent displaylayer). FIG. 19 shows how mask 88 may be formed from a through-holeaperture in a relatively thick display layer (e.g., a pixel definitionlayer or other opaque display layer). In the FIG. 19 configuration, thewidth W of opening 90 is smaller than the thickness T of the opaquelayer forming mask 88. Masks such as the masks of FIGS. 17, 18, and 19may be used with or without one or more lenses such as lens 86. Theangular light filters formed using lenses 86 and/or masks 88 may eachoverlap and be aligned with a respective light detector (e.g., a pixel Pwith a photodetector) or may otherwise be used to help restrict theangular acceptance of the light detectors in the optical touch sensor.

Optical touch sensor measurements may be gathered during periods of timein which image light is not being output from display 14 or may begathered during the periods of time in which image light is displayed.

Consider, as a first example, an arrangement of the type shown in thetiming diagram of FIG. 20. In FIG. 20, time periods P1 correspond totime periods in which image pixels are outputting light for an imagethat is being viewed by a user and periods P2 are periods of time inwhich light detecting pixels for an optical touch sensor are gatheringsensor measurements. In one illustrative arrangement, all image pixelsin display 14 emit light only during periods P1 (e.g., image frames) andall light detecting pixels in display 14 make sensor measurements todetect scattered light from finger 34 only during periods P2 betweenperiods P1. In this type of operation, optical touch sensor illuminationis provided using light sources separate from the image pixels indisplay 14. In another illustrative arrangement, the image outputoperations of periods P1 and the light sensing operations of periods P2may overlap. When optical touch sensor illumination is being produced byimage pixels, the optical touch sensor may use image pixel modulationand/or knowledge of the intensity of image pixel output at each locationon display 14 to help analyze the detected scattered light. When opticaltouch sensor illumination is being provided from a light sourceproducing totally internally reflected light in display cover layer14CG, both the optical touch sensor light from the light source andimage pixel light may be produced during periods P2.

As shown in FIG. 21, images may be output during periods P1 (e.g., imageframes) while optical touch sensor measurements are made duringoverlapping time periods P2. As an example, light for illuminatingfinger 34 may be produced by image pixels during periods P2 or may beproduced during periods P2 by separate light sources (e.g., infraredlight-emitting diodes or other infrared light-emitting pixels that areseparate from the image pixels of display 14). Light-sensing pixels(e.g., infrared light-sensing pixels or visible light sensing pixels)may gather scattered light measurements during periods P2. In somearrangements (e.g., when infrared light is being used for optical touchsensing), light-sensing pixels may have optical filters that blockvisible light and pass infrared light to reduce potential visible lightinterference from image pixels. In configurations in which visible lightis used to illuminate finger 34, image frame information on which imagepixels are active across display 14 may be used in determining how muchvisible light is expected to be scattered by finger 34 at a givenlocation on display 14.

Signal modulation techniques (e.g., modulation of emitted light with aknown pattern over time, at a predetermined frequency, etc. andcorresponding demodulation of sensed light) may be used to help extractoptical touch sensor signals from detected ambient light signals and/ormeasured signals associated with stray image light. For example, emittedlight may be modulated at a particular frequency and detected lightsignals demodulated (synchronously) at the same frequency. In this way,external optical interference from ambient light sources and internaloptical interference (e.g., interference from stray display light, whichmay be produced during sensing periods in some embodiments) may berejected.

As shown in the example of FIG. 22, display panel 14P may include pixelsP that each include a respective diode 92. Pixels P may use diodes 92 aslight sources. For example, diodes 92 may be forward biased to serve asimage pixels that emit visible light that forms images for a user.Emitted light 46 from diodes 92 may also serve as illumination for anoptical touch sensor (e.g., light 46 may be backscattered in thepresence of finger 34 on display 14 to form backscattered light 48).Diodes 92 may be light-emitting diodes (thin-film organic light-emittingdiodes or crystalline semiconductor dies) or laser diodes. In someconfigurations, diodes 92 may be configured to emit infrared light. Intotal internal reflection optical touch sensor arrangements,illumination for finger 34 may, if desired, be supplied by providingdisplay cover layer 14CG with edge illumination, as described inconnection with FIG. 8.

Some or all of diodes 92 may be reversed biased to serve asphotodetectors for the optical touch sensor. The photodiodes may, as anexample, extend in an array across display 14, so that the photodiodesmay measure and thereby determine the location of backscattered light 48from finger 34.

The diodes 92 that serve as photodetectors in the optical touch sensormay be used exclusively as optical touch sensor light detectors or maysometimes be forward biased to emit light for images and/or opticaltouch sensor illumination and sometimes reverse biased to serve asphotodetectors for the optical touch sensor. Light-detecting diodes 92may, as an example, sometimes emit visible images light (e.g., whileserving as image pixels) and may sometimes detect backscattered light 48(see, e.g., pixels P′, in which diodes 92 is configured both to emitlight 46 and to detect light 48). In arrangements in which diodes 92 canserve both as light emitters and light detectors, the use of additionaloptical components to form the optical touch sensor (e.g., additionallight-emitting devices and/or light sensors) may be reduced oreliminated.

If desired, additional components for optical touch sensor pixels may beformed above or below an array of pixels. Consider, as an example, thecross-sectional side view of display panel 14P of FIG. 23. In theexample of FIG. 23, pixels P″ are formed in a single layer (e.g., alayer of thin-film pixels on a flexible or rigid display panel substrateas described in connection with pixel P of FIG. 12 or a layer ofcrystalline semiconductor dies that form pixels as described inconnection with pixels P of FIG. 13). In an illustrative configuration,pixels P″ in display panel 14P of FIG. 23 form a layer of image pixelsthat emit visible light for producing images viewed by a user of device10. Configurations in which pixels P″ have diodes that can be reversebiased to form photodetectors for an optical touch sensor and/or thathave infrared diodes for providing optical touch sensor illumination mayalso be used.

As shown in FIG. 23, optical components 94 may be located above and/orbelow pixels P″. Optical components 94 may be based on diodes (e.g.,diodes that emit light such as light-emitting diodes and laser diodes,diodes that detect light such as photodiodes, and/or diodes that may beforward biased to emit light and reversed biased to detect light).Diodes and/or other structures forming components 94 may be thin-filmdiodes (e.g., organic light-emitting diodes, organic thin-film diodesthat serve as photodetectors, etc.) and/or may be formed fromcrystalline semiconductor dies. Components 94 may be mounted on one ormore substrates such as substrate 62 of FIG. 13, may be mounted on ashared substrate with the structures of pixels P″, and/or may beotherwise incorporated into display panel 14P.

In some configurations, light-emitting components 94 may be locatedabove pixels P″ and light-detecting components 94 may be located belowpixels P″. In other configurations, light-emitting components 94 may belocated below pixels P″ and corresponding light-detecting components forthe optical touch sensor may be located above pixels P″. Arrangements inwhich some light-emitting components 94 are mounted above and belowpixels P″ and/or in which some light-sensing components 94 are mountedabove and below pixels P″ may also be used. Pixels P″ and/or components94 may operate using visible and/or infrared light.

In arrangements in which optical touch sensor components 94 are formedabove pixels P″, the substrate on which components 94 are mounted may betransparent to light emitted and/or detected by pixels P″. Inarrangements in which optical touch sensor components 94 are formedbelow pixels P″, the anodes of pixels P″, the pixel definition layerused in forming pixels P″, and/or other structures of the pixel arrayforming pixels P″ may be sufficiently transparent (by using materialsthat pass infrared and/or visible light, by forming openings, etc.) toallow components 94 to operate through the layer of pixels P″. As anexample, pixels P″ may be contained in a thin-film organiclight-emitting diode display panel with anodes that are sufficientlytransparent to pass infrared light for the optical touch sensor andcomponents 94 may include infrared light-emitting diode dies andinfrared photodetector dies mounted on a substrate layer that is belowpixels P″. In total internal reflection optical touch sensors, light(e.g., infrared light or visible light) for the optical touch sensor maybe emitted into display cover layer 14CG and backscattered light 48 maybe detected by photodetectors (e.g., light-sensing components 94 aboveand/or below pixels P″).

If desired, backlight pixels may be used in forming an optical touchsensor. Consider as an example, display panel 14P of FIG. 24. In thisexample, an array of image pixels P may be formed using liquid crystaldisplay pixels in a liquid crystal display panel. Backlight unit 98 mayemit backlight illumination that passes through the liquid crystaldisplay panel formed by pixels P. Backlight unit 98 may have an array ofbacklight pixels BP that can be locally dimmed to enhance image contrast(e.g., backlight unit 98 may be a direct-lit backlight unit thatsupports local dimming). Backlight pixels BP may contain opticalcomponents 96. Components 96 may include white-light backlight pixels orother backlight pixels that produce backlight illumination for pixels P.Components 96 may also include light sources (visible and/or infrared)and/or light detectors (visible and/or infrared) for forming an opticaltouch sensor. As an example, each backlight pixel BP may have abacklight illumination component such as a light-emitting diode thatemits backlight illumination (e.g., white light backlight illumination)may have an infrared light source (e.g., an infrared light-emittingdiode or infrared laser), and may each have an infrared light detector(e.g., an infrared photodetector). The infrared light sources anddetectors in this type of arrangement may be mounted on a commonsubstrate with the backlight illumination light-emitting diodes and/ormay be mounted on other substrates (above and/or below the backlightillumination light-emitting diodes). Arrangements in which visible lightfrom the backlight light-emitting diodes is used to produce opticaltouch sensor illumination (e.g., light 46 that illuminates a user'sfinger to produce backscattered light 48) may also be used.

Although sometimes described in the context of an arrangement in whichtouch sensor operation occurs through light that is propagating withindisplay cover layer 14CG primarily at a single angle, the light sourcemay emit light into display cover layer 14CG at multiple distinct angles(e.g., an angle A1 and different angle A2). In this type of arrangement,a first object with a first refractive index nfirst may locally defeattotal internal reflection for light at angle A1 while not locallydefeating total internal reflection for light at angle A2, whereas asecond object with a second refractive index nsecond that is greaterthan the first refractive index may locally defeat total internalreflection for both light at angle A1 and light at angle A2. Because thefirst and second objects interact differently with the optical touchsensor, the touch sensor can discriminate between the first and secondobjects. This allows device 10 to respond differently to input from thedifferent types of objects. As an example, in a drawing application,lines may be drawn with a first thickness when the first object is movedacross layer 14CG, whereas lines may be drawn with a second thicknesswhen the second object is moved across layer 14CG. The first and secondobjects may be any suitable objects (one or more different types ofstylus, a finger, and/or other objects). If desired, light at each anglemay be associated with a different respective color and dedicated setsof detectors (each responsive to a different color) may be used.

The foregoing is merely illustrative and various modifications can bemade to the described embodiments. The foregoing embodiments may beimplemented individually or in any combination.

Table of Reference Numerals 10 Electronic device 12 Input-output devices14 Display 18 Sensors 16 Control circuitry 22 Housing P, P-1, P-2,Pixels 30 Interior Region P′, P″ F Front face 36 Components 32 Exteriorregion 38 Substrate 14P Display Panel 14CG Display Cover Layer 34 Finger40 Viewer 42 Direction 44 Backlight unit 46, 46′, 48, Light 50 Layer 48′A Angle 60 Water n Surface Normal 52 Light Source 54 Collimator 56 Prism70 Pixel Definition 66 Anode Layer 68 Organic Layers 64 Thin-film Layers62 Substrate 72 Encapsulation Layers 74 Semiconductor Dies 78Index-Matching Layer 80 Opening 76 Mask 86 Lens 88, 88′ Mask 90 Opening82 Filter 84 Transparent Layer P1, P2 Time Periods 92 Diodes 94 OpticalComponents BP Backlight Unit Pixels 98 Backlight Unit 96 OpticalComponents

What is claimed is:
 1. An electronic device configured to gather touchinput from a finger, comprising: a display having a display cover layerwith a surface; and an optical touch sensor having a light source andlight detectors, wherein the light source is configured to emit lightinto the display cover layer that is guided within the display coverlayer by total internal reflection while the surface of the displaycover layer is immersed in water and wherein total internal reflectionis locally defeated to scatter the light towards the light detectorswhen the surface is contacted by the finger.
 2. The electronic devicedefined in claim 1 wherein the light detectors are arranged in an arrayextending across the display.
 3. The electronic device defined in claim1 wherein the display has an array of light-emitting diodes configuredto display an image.
 4. The electronic device defined in claim 3 whereinthe light source is overlapped by the array of light-emitting diodes. 5.The electronic device defined in claim 1 wherein the display cover layerhas an edge surface and wherein the light source is configured to emitlight into the edge.
 6. The electronic device defined in claim 5 whereinthe light source has a light-emitting diode, a collimator, and anoptical coupler that is coupled between the collimator and the edgesurface.
 7. The electronic device defined in claim 1 wherein the displayhas an array of liquid crystal display pixels.
 8. The electronic devicedefined in claim 1 wherein the finger has a first refractive index andwherein the display cover layer has a second refractive index that isless than the first refractive index.
 9. The electronic device definedin claim 8 wherein the display cover layer comprises glass.
 10. Theelectronic device defined in claim 1 wherein the finger has a firstrefractive index and wherein the display cover layer has a secondrefractive index that is greater than the first refractive index. 11.The electronic device defined in claim 10 wherein the display coverlayer comprises sapphire.
 12. The electronic device defined in claim 10wherein the surface has a surface normal, wherein the light source isconfigured to emit the light into an edge surface of the display coverlayer at an angle with respect to the surface normal, and wherein theangle has a value between 49.08° and 61.72°.
 13. The electronic devicedefined in claim 12 wherein the light source has a light-emitting diode,a collimator, and an optical coupling structure coupled between thecollimator and the edge surface.
 14. The electronic device defined inclaim 13 wherein the light-emitting diode comprises an infraredlight-emitting diode.
 15. The electronic device defined in claim 13wherein the light-emitting diode comprises a visible light-emittingdiode.
 16. The electronic device defined in claim 1 wherein the lightsource comprises an infrared light source, wherein the emitted lightcomprise infrared light, wherein the display comprises an array oforganic light-emitting diode pixels with anodes that transmit at leastsome of the infrared light when the infrared light is scattered towardthe light detectors by the finger, and wherein the light detectorscomprise a layer of infrared photodetectors overlapped by the array oforganic light-emitting diode pixels.
 17. The electronic device definedin claim 1 wherein the light source is configured to modulate theemitted light and wherein the light detectors are configured tosynchronously demodulate the scattered light.
 18. The electronic devicedefined in claim 1 further comprising a mask with an opening overlappingeach of the light detectors to form an angular light filter for thatlight detector.
 19. The electronic device defined in claim 18 furthercomprising a lens overlapping each opening.
 20. The electronic devicedefined in claim 1 wherein the display has image pixels configured toemit image light and wherein the light detectors are separate from theimage pixels and do not emit light.
 21. The electronic device defined inclaim 1 wherein the display has an array of diodes configured to emitimage light and wherein at least some of the diodes serve as the lightdetectors.
 22. The electronic device defined in claim 1, wherein eachlight detector is overlapped by an angular light filter.
 23. Theelectronic device defined in claim 22, wherein each angular light filterincludes a mask with an opening, a lens that overlaps the opening, and atransparent layer between the mask and the lens.
 24. The electronicdevice defined in claim 22, wherein each angular light filter includes afirst mask with a first opening, a second mask with a second openingthat overlaps the first opening, and a transparent layer between thefirst mask and the second mask.